Filling the Gap of Utterance-aware and Speaker-aware Representation for Multi-turn Dialogue

A multi-turn dialogue is composed of multiple utterances from two or more different speaker roles. Thus utterance- and speaker-aware clues are supposed to be well captured in models. However, the existing retrieval-based modeling, pre-trained language models (PrLMs) as encoder represent dialogues coarsely by taking pairwise history candidate response a whole, hierarchical information on either utterance interrelation roles coupled such representations not addressed. In this work, we propose novel model fill gap modeling effective utterance-aware entailed history. detail, decouple contextualized word masking mechanisms Transformer-based PrLM, making each only focus words current utterance, other utterances, (i.e., sender receiver), respectively. Experimental results show that our method boosts strong ELECTRA baseline substantially four public benchmark datasets, achieves various new state-of-the-art performance over previous methods. series ablation studies conducted demonstrate effectiveness method.